The computer industry is moving toward fast, packetized, serial input/output (I/O) bus architectures, in which computing hosts and peripherals are linked by a switching network, commonly referred to as a switching fabric. A number of architectures of this type have been proposed, culminating in the “InfiniBand™” (IB) architecture, which has been advanced by a consortium led by a group of industry leaders (including Intel, Sun Microsystems, Hewlett Packard, IBM, Compaq, Dell and Microsoft). The IB architecture is described in detail in the InfiniBand Architecture Specification, Release 1.0, which is available from the InfiniBand Trade Association at www.infinibandta.org and is incorporated herein by reference.
InfiniBand uses an “absolute,” credit-based link-level flow control mechanism in order to prevent loss of packets due to buffer overflow by the receivers at either end of a link. This mechanism is described in section 7.9 of the above-mentioned specification (pages 175-179). Any given physical link between two switches in an IB fabric may carry up to fifteen logical links, referred to in IB parlance as “Virtual Lanes” (VLs). The switch port at each end of each physical link includes a transmitter and receiver, for sending packets to and receiving packets from the corresponding port at the other end of the link. The receiver controls packet flow over each of the VLs by providing the transmitter with “credit limits,” indicating the total amount of data that the transmitter has been authorized to send. When the credit limit for a given VL has been exhausted, the transmitter is not permitted to send any more data over that VL until it has received a flow control packet from the receiver, indicating that additional credit has become available.
The credit limits are calculated by the receiver in such a manner as to guarantee that there will always be room available in the receive buffer for any packets sent by the transmitter within the applicable credit limits. This means that the receiver must leave an adequate amount of vacant buffer space for every one of the VLs that it serves. In order to maximize utilization of the bandwidth available on the physical link, the receiver preferably updates the transmitter's credit limits continually (and sends the appropriate flow control packets to the transmitter, while the transmitter is transmitting data), rather than waiting until the transmitter has used up its credit limit. It would thus appear that very large buffers are required at all of the switch ports in order to maintain efficient, wire-speed communications throughout the fabric.